Comparing Producer-Consumer Implementations in Go, Rust, and C

The producer-consumer model is a classic synchronization problem, exploring the division of labor between threads. Producers are responsible for creating or yielding items of some kind, while consumers are responsible for processing these items. The producer-consumer model is present in a wide array of multithreaded applications. It is commonly used to read data from high-throughput streaming APIs, and underlies the design of popular streaming data services such as RabbitMQ and Apache Kafka. Producer-consumer is a model frequently used in other data-intensive tooling, frequently appearing in any tooling used to ingest and process large quantities of data. Given the prevalence of the producer-consumer model, we sought to examine implementations of this model across systems programming languages. The producer-consumer model offered a valuable opportunity to explore and compare concurrency tools among systems languages, and implementing producer-consumer in multiple languages could reveal the utility of using certain languages for data-intensive processing. In our research, we seek to answer two questions: (1) how did concurrency tools such as threads and shared/concurrent queues broadly differ across systems languages and (2) how did ease of implementation and performance compare between these languages? In this paper, we implement and benchmark producer-consumer models in the increasingly prominent systems languages Go and Rust, in addition to the mainstay language C. We analyze the ease of writing multithreaded code in these languages, and the subsequent performance of our implementations on common producer-consumer use cases. Our code can be found in our GitHub repo at https://github.com/gabbifish/producer-consumer.